1. Field of the Invention
This invention is generally related to an apparatus and method of capturing images of an object, e.g. image mosaicing, and more particularly is related to an apparatus and method of capturing images by a combination of acceleration sensors and magnetic sensors, and optionally angular speed sensors.
2. Background of the Invention
It is known to capture a three-dimensional image of an object, for example if an image mosaicing operation is desired, or to represent a three-dimensional image on a computer. Systems for generating three-dimensional images of an object are known. These systems can be viewed as either active processes or passive processes.
One known active process is a wave propagation method in which a camera calculates a distance to an object based on a measured propagation time of a reflection wave from the object, for example by a reflective light wave, radio wave, or sound wave. Another active process is a light propagation method in which a light source located at a predetermined position irradiates a slit light or a spot light onto an object. A detector then detects the shape of the object by measuring a distortion of the slit light or the spot light.
Each of these active processes, however, suffers from a drawback in that it is difficult to miniaturize the apparatuses to implement such processes and have the apparatuses still make the required measurements.
The passive types of processes include a binocular stereo process and a motion stereo process. In a binocular stereo process plural cameras are located at predetermined positions and attitudes relative to an object, and these plural cameras each capture an image of the object. Then, a correspondence of characteristic points or ranges between each image of the object is carried out. The three-dimensional shape of the object can then be calculated by utilizing the principle of a triangular surveying based on the plural images detected by the plural cameras. In the motion stereo process a single camera captures images of the object, but the single camera is moved to capture successive images of the object from different points. Then, a correspondence of characteristic points or ranges between all of the captured successive images of the object are carried out as the camera moves. The three-dimensional shape of the object can then be calculated based upon the correspondence of the characteristic points or ranges.
However, these passive processes of the binocular stereo process and motion stereo process also suffer from drawbacks. More particularly, in each of these systems an error caused by noise of the correspondence of the characteristic points or ranges between each image arises. Further, if there is a low disparity between sequential images, a large measurement error may arise. Further, in the motion stereo process a repetitive calculation of a complex non-linear equation must be executed because there are no predetermined relationships between the position of the object and the camera.
Certain Japanese Laid-Open Patents have addressed problems in capturing three-dimensional images of an object.
Japanese Laid-Open Patent No. 07-181024 discloses an image pick-up apparatus, e.g. a camera, which includes a stroke detector which utilizes an acceleration sensor and an angle speed sensor. The stroke detector detects the stroke of the camera. The system then evaluates the stroke with respect to a translational component and a rotational component of the stroke. Then, a reconstruction of the three-dimensional shape is executed by a result of searching for correspondence points.
However, this system disclosed in Japanese Laid-Open Patent No. 07-181024 has a drawback in that the calculation of the stroke requires an integration. Therefore, an error component of the stroke is accumulated.
Japanese Laid-Open Patent No. 09-81790 sets forth a system in which a movement of a camera is detected by an angular speed sensor and an acceleration sensor. A high speed restoration of the three-dimensional object is carried out while preventing a disappearance of a characteristic point by revision of a light axis direction such as a different light axis from a viewpoint to not intersect.
However, the device disclosed in Japanese Laid-Open Patent No. 09-81790 suffers from a drawback in that the device has a complex structure because an optical axis is changed.
Japanese Laid-Open Patent No. 09-187038 sets forth a system in which images of a three-dimensional object are captured from plural viewpoints so that parts of the images are duplicated with each other. An image pick-up parameter is then calculated by a signal corresponding to a position of a lens and a signal corresponding to a position of the camera. A distance distribution to the object is then calculated whenever a shutter is pushed. The three-dimensional model of the object is then generated based upon the distance distribution.
However, this system disclosed in Japanese Laid-Open Patent No. 09-187038 also suffers from a drawback in that the calculation of the stroke requires an integration. Therefore, again the error component of the stroke is accumulated.
Accordingly, it is one object of the present invention to provide a novel apparatus for generating a three-dimensional image which can overcome the drawbacks in the background art.
Another object of the present invention is to provide a novel three-dimensional image forming apparatus which can generate a three-dimensional image of an object with a simple and accurate structure.
Another object of the present invention is to provide a novel three-dimensional image forming apparatus which can generate a three-dimensional image of an object while also compensating for any disturbances in a detector.
The present invention achieves these and other objects by providing a novel image capturing apparatus for capturing an image of an object. The novel image capturing apparatus includes a correspondence detector which detects a correspondence of characteristic points of an object based upon captured images of the object. A motion detector detects a motion of the image capturing device. The motion detector includes magnetic sensors. Further, a shape calculator calculates a shape of the object based upon the captured image data, captured attitude information, a translation component, and the correspondence of the characteristic points.
Further, in the novel image capturing apparatus of the present invention the motion detector may further include acceleration sensors and angle speed sensors. A disturbance detector may further be provided to detect a disturbance in acceleration signals output by the acceleration sensors and a disturbance in magnetic signals output by the magnetic sensors. In this situation, if a disturbance arises in the acceleration signals, an attitude of the image capturing device can be based on signals output by the magnetic sensors and signals output by the angle speed sensors. Similarly, if a disturbance arises in the magnetic signals output by the magnetic sensors, an attitude of the image capturing device can be determined utilizing the acceleration signals output by the acceleration sensors and the angle speed signals output by the angle speed sensors.